181 research outputs found
A moment-matching Ferguson and Klass algorithm
Completely random measures (CRM) represent the key building block of a wide
variety of popular stochastic models and play a pivotal role in modern Bayesian
Nonparametrics. A popular representation of CRMs as a random series with
decreasing jumps is due to Ferguson and Klass (1972). This can immediately be
turned into an algorithm for sampling realizations of CRMs or more elaborate
models involving transformed CRMs. However, concrete implementation requires to
truncate the random series at some threshold resulting in an approximation
error. The goal of this paper is to quantify the quality of the approximation
by a moment-matching criterion, which consists in evaluating a measure of
discrepancy between actual moments and moments based on the simulation output.
Seen as a function of the truncation level, the methodology can be used to
determine the truncation level needed to reach a certain level of precision.
The resulting moment-matching \FK algorithm is then implemented and illustrated
on several popular Bayesian nonparametric models.Comment: 24 pages, 6 figures, 5 table
On the sub-Gaussianity of the Beta and Dirichlet distributions
We obtain the optimal proxy variance for the sub-Gaussianity of Beta
distribution, thus proving upper bounds recently conjectured by Elder (2016).
We provide different proof techniques for the symmetrical (around its mean)
case and the non-symmetrical case. The technique in the latter case relies on
studying the ordinary differential equation satisfied by the Beta
moment-generating function known as the confluent hypergeometric function. As a
consequence, we derive the optimal proxy variance for the Dirichlet
distribution, which is apparently a novel result. We also provide a new proof
of the optimal proxy variance for the Bernoulli distribution, and discuss in
this context the proxy variance relation to log-Sobolev inequalities and
transport inequalities.Comment: 13 pages, 2 figure
Approximating predictive probabilities of Gibbs-type priors
Gibbs-type random probability measures, or Gibbs-type priors, are arguably
the most "natural" generalization of the celebrated Dirichlet prior. Among them
the two parameter Poisson-Dirichlet prior certainly stands out for the
mathematical tractability and interpretability of its predictive probabilities,
which made it the natural candidate in several applications. Given a sample of
size , in this paper we show that the predictive probabilities of any
Gibbs-type prior admit a large approximation, with an error term vanishing
as , which maintains the same desirable features as the predictive
probabilities of the two parameter Poisson-Dirichlet prior.Comment: 22 pages, 6 figures. Added posterior simulation study, corrected
typo
Bayesian nonparametric dependent model for partially replicated data: the influence of fuel spills on species diversity
We introduce a dependent Bayesian nonparametric model for the probabilistic
modeling of membership of subgroups in a community based on partially
replicated data. The focus here is on species-by-site data, i.e. community data
where observations at different sites are classified in distinct species. Our
aim is to study the impact of additional covariates, for instance environmental
variables, on the data structure, and in particular on the community diversity.
To that purpose, we introduce dependence a priori across the covariates, and
show that it improves posterior inference. We use a dependent version of the
Griffiths-Engen-McCloskey distribution defined via the stick-breaking
construction. This distribution is obtained by transforming a Gaussian process
whose covariance function controls the desired dependence. The resulting
posterior distribution is sampled by Markov chain Monte Carlo. We illustrate
the application of our model to a soil microbial dataset acquired across a
hydrocarbon contamination gradient at the site of a fuel spill in Antarctica.
This method allows for inference on a number of quantities of interest in
ecotoxicology, such as diversity or effective concentrations, and is broadly
applicable to the general problem of communities response to environmental
variables.Comment: Main Paper: 22 pages, 6 figures. Supplementary Material: 11 pages, 1
figur
Dirichlet process mixtures under affine transformations of the data
Location-scale Dirichlet process mixtures of Gaussians (DPM-G) have proved
extremely useful in dealing with density estimation and clustering problems in
a wide range of domains. Motivated by an astronomical application, in this work
we address the robustness of DPM-G models to affine transformations of the
data, a natural requirement for any sensible statistical method for density
estimation and clustering. First, we devise a coherent prior specification of
the model which makes posterior inference invariant with respect to affine
transformations of the data. Second, we formalise the notion of asymptotic
robustness under data transformation and show that mild assumptions on the true
data generating process are sufficient to ensure that DPM-G models feature such
a property. Our investigation is supported by an extensive simulation study and
illustrated by the analysis of an astronomical dataset consisting of physical
measurements of stars in the field of the globular cluster NGC 2419.Comment: 36 pages, 7 Figure
Bayesian optimal adaptive estimation using a sieve prior
We derive rates of contraction of posterior distributions on nonparametric
models resulting from sieve priors. The aim of the paper is to provide general
conditions to get posterior rates when the parameter space has a general
structure, and rate adaptation when the parameter space is, e.g., a Sobolev
class. The conditions employed, although standard in the literature, are
combined in a different way. The results are applied to density, regression,
nonlinear autoregression and Gaussian white noise models. In the latter we have
also considered a loss function which is different from the usual l2 norm,
namely the pointwise loss. In this case it is possible to prove that the
adaptive Bayesian approach for the l2 loss is strongly suboptimal and we
provide a lower bound on the rate.Comment: 33 pages, 2 figure
Sub-Weibull distributions: generalizing sub-Gaussian and sub-Exponential properties to heavier-tailed distributions
We propose the notion of sub-Weibull distributions, which are characterised
by tails lighter than (or equally light as) the right tail of a Weibull
distribution. This novel class generalises the sub-Gaussian and sub-Exponential
families to potentially heavier-tailed distributions. Sub-Weibull distributions
are parameterized by a positive tail index and reduce to sub-Gaussian
distributions for and to sub-Exponential distributions for
. A characterisation of the sub-Weibull property based on moments and
on the moment generating function is provided and properties of the class are
studied. An estimation procedure for the tail parameter is proposed and is
applied to an example stemming from Bayesian deep learning.Comment: 10 pages, 3 figure
The fine print on tempered posteriors
We conduct a detailed investigation of tempered posteriors and uncover a
number of crucial and previously undiscussed points. Contrary to previous
results, we first show that for realistic models and datasets and the tightly
controlled case of the Laplace approximation to the posterior, stochasticity
does not in general improve test accuracy. The coldest temperature is often
optimal. One might think that Bayesian models with some stochasticity can at
least obtain improvements in terms of calibration. However, we show empirically
that when gains are obtained this comes at the cost of degradation in test
accuracy. We then discuss how targeting Frequentist metrics using Bayesian
models provides a simple explanation of the need for a temperature parameter
in the optimization objective. Contrary to prior works, we finally
show through a PAC-Bayesian analysis that the temperature cannot be
seen as simply fixing a misspecified prior or likelihood
Clustering Milky Way's Globulars: a Bayesian Nonparametric Approach
International audienceThis chapter presents a Bayesian nonparametric approach to clustering , which is particularly relevant when the number of components in the clustering is unknown. The approach is illustrated with the Milky Way's glob-ulars, that are clouds of stars orbiting in our galaxy. Clustering globulars is key for better understanding the Milky Way's history. We define the Dirichlet process and illustrate some alternative definitions such as the Chinese restaurant process, the Pólya Urn, the Ewens sampling formula, the stick-breaking representation through some simple R code. The Dirichlet process mixture model is presented, as well as the R package BNPmix implementing Markov chain Monte Carlo sampling. Inference for the clustering is done with the variation of information loss function
- …